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ABSTRACT " t 4 

p A computer natural language system called HWIM (Hear 
Vtytet I^Mean). is described in this report. Noting that the system • 
•accepts either typed or spoken inputs and produces both typed and 
spoken, resp9nses f the report proposes HWI14 as an example of a 
<relatij£ely**tomplete language system illustrating how* the many 
components of language processing interact. The report focuses an 
-HWIM'<s discourse processing capabilities and on the integration- of 
diverse types of knowledge for the' purpose 6f comprehending and 
producing natural language. ^FL) t ' 
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HWIM: A Computer Model of Language 
\ Comprehension and Production , 

People design computer programs to understand natural 
Language for two^ reasons. One is to make computers more useful 
by facilitating_ communication between the computer and computer, 
users.' The other is that, the 'design of such programs can, inform^ 

to 

theories* of human language use. The program then becomes a model 
for human language comprehension or production. Decisions made 
in the design process in order to enhance efficiency, or siihply 
to make the program succeed, suggest -characteristics of the 
processes, humans carry out when performing similar .taskS. Also>, * 



the extent to which a theory" of language Qse actually^ "works tt: 
when it is implemented as a computer model, is .one measure of \ its % 
correctness.* Moreover, the act^ of expressing a, -theory in/ a* 
computer program forces us~to be explicit and unambiguous about 
what the theory is. By examining the ^development, the structure,' 
and the operation of computer programs that successfully usq 

o * • • r . « 

natural language, one may gain insights into human use of natural 
language, including reading and writing, speaking and listening. 

r ^ i 

t This paper discusses some general issues of. language 4 
comprehension and production - through examination of a complete 
natural language system called "HWIM" (for "Hear what I meant.). 
HWIM was * developed over a, five year period a£°'the Bolt Beranek 
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and Newman speech understanding system. ' It was designed to 
understand natural language (typed or spoken); to answer 
questions, per form calculations, and. maintain a data base; arid to 
respond in natural language (typed and spoken). Both its inputs 
and outputs used a« relatively rich grammar with /a 1000 word 
vocabulary. Utterances were assumed to be part of an on-going 
dialogue so that HWIM .had to have a model of bdth the discourse 
and the user. This paper focuses on the component of the system 1 
embodying semantic and pragmatic knowledge. ' -A fuller treatment- 
of the system "can be found in Bjruce (ife press) and in Woods, 
Bates, Brown, Bruce, Cook, .Klovsfad, Makhoul, Nash-Webber, 
Schwartz, ,Wolf, and Zue (Note 9). For discussions of other 
speech understanding systems, see Erman, Hayes-RothT Lesser, and 
'Redfly -ft^8 (f) ; tea (1980); and Walker (1978). ' 

HWIM 1 s Job Responsibilities 

HWIM was de-signed to serve as an assistant for the manager 
of a travel budget. The task of- the system was to' assist' the 
travel budget manager, helping *to .record the trips taken or 
propose^and to produce such summary information as the total 
money allocated to various budget items. In order to carry out 
its duties, -HWIM ne^cfi^S to converse with ,the travel budget' 
manager.. -Though their conversations were simplified* relati^ to' 
natural inter-personal conversation, the/design of the system can 
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help in fprmulating more -general' models of- language 
understanding. Some, salient features of HWIM are the following: 

*. o the maintenance of dynamic models of the discourse, 
the task, the user, and specific facts about the 
world • . 

o the use of such k-nowledg.e to / constrain "possible 
interpretations of. utterances, thus increasing the - ' 
.\ likelihood bf successful, understanding . 

. o the use of the same ^knowledge in genera tinq both 
i t written .and' spoken responses; a formalism for . 



/ • . • ... 

;. expressing response generation rules which provides 
j / t a framework for tlje application, of semantic and 
discourse level knov/ledge 



o facilities for making "the, system's own knowledge 
( state known to* the travel budget manager \ \ 

o inference mechanisms and data" base* structures^ that 
.facilitate freer expression of ' commarids and 
N ^ . quest ipns By the travel budget manager 



. • Steps in Processing 

In this * section we see a',trace..of HWIM 1 processing, * -Jihe ' 
sentences .shown here" would normally occur as part of' an ongoing' 
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.dialogue. Whiie their interpretations and resulting responses 
might be* significantly* different in. context, . what . carv be seen 
here is- the flow of control within the system, the types of 
interpretations produced, the inference* capabilities, and 
response generation. The structure of HWIM is shown in Figure 1 
(an<J discussed' further 'in Appendix A). ~ 

'Wl^en HWIM engage^ in a text dialogue, the system .reduces to 
just two processes, T*dp and Syntax. In that mode Syntax is a 
subprocess jto Trip, ^though it may itself invoke Trip as its own 
subprocess to evaluate tests on arcs in the grammar.' To .see one 
possible-' flow of control, (Figure 2), consider the Sentence - 

• f • • • / • 

Entejr a, trip for Jerry Wolf to New York. s 

* • 

Trip reads the input and does a morphological' analysis of each^ 
word. .it then calls Syntax to p.arse and interpret the sentence. 
During processing, Syntax may encounter a test, e.g.; "Does . the/ 
name 'Jerry' Wolf 1 * denote a knowh person?", whicfo is t9 be 
performed by Trap. .Control passes to Tripj8|^ich answers ^y e s," 
and then, back to Syntax " again. When the parse is ^omplete, 
control returris to Trip.. Ambiguity, in the input can cause a 
question to be formulated for the travel budget mana^r. The 
manager Js 1 respons^/ould cause a return ^feo Syntax, and so on to / 
the end ofr the session'.' * . *' 
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1 . "Enter a trip for . ' 
Jerry Waif to New York."* 



Z. PARSE 
v ((ENTER A TRIP/..)) 




(BUILD: TRIP -) ~> ) 



igure 2, 



One possible flow of control for a dialogue 
interchange. ; • . % > 
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• , To see the steps, HWIM takfl*Tn reading, comprehending, and' 
then responding to some* text, imagine that* the travel budget, 
mfenager types: ^ 

When did Bill go to Mexico? 

* . < 

The program makes a, fffst pass on the input, converting the words' 
as typed into words as they appear in its dictionary and removing' 
' punctuation. For example, "$23.16" would becon\'e "twenty - th ree 
dollar-s and sixteen cent-s" . In this case the modified word 
list is simply 

* ♦ \ 1 • 

.(WHEN DID BILL GO TO MEXICO) 

For spoken inputs .HWIM has to consider, many possible -word 
* lists because the inpiit is ambiguous. This could be viewed, as 
. - analogous to problems faced by a person in reading (see 
- Rumelhart, 1977; Woods, 1980b). For typed' inputs", HWIM is a 
perfect decoder; thus„ only one word list needs to be considered. 
In j ord.er to simplify the discussion, the example here will focus 
on a single word list. 

The parser in HWIM uses a pragonatic grammar , which contains 
significant static knowledge of ? the travel^budget management 
, domain. Collapsing the task specific knowledge into the gramm 
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was viewed as an expedient to gain computational (processing) 
efficiency. It helped to constrain, parses early and .enabled 
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simultaneous parsing and semantic interpretation. This is in 
* • 

contrast with the two phase method used in the LUNA& parser 
(Woods, Kaplan & Nash-Webber , Note :10) which produced purely, 
syntactic^ parse tree^s that were subsequently given to* a semantic 
interpreter" to t^nsforra into executable? interpretations. More 
recently, techniques such as* the use^ of . cascaded ATN 1 s (Woods, 
1980a) ^ as in the RUS parser ( Bob rq&+ Note 2), have been developed 
to- gain much of the efficiency Of pragmatic or semantic grammars- 

while maintaining' a clean separation between general syntactic 

• / 

knowledge and task specific knowledge. 

For the example sentence, the parser derives .the 'syntactic' 
structure: . " * ' 



1 QADV WHEN 
SUBJ NPR BILL 
AUX TN6 PAST 

VOICE ACTIVE 
VP V GO 

PP PREP. TO 

NP NpR MEXICO' 
ADV THEN 



Taking advantage of the. semantic knowledge in the grammar, we get 
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in addition: 



[FOR: THE AQ0Q9 / (FIND: LOCATION (COUNTRY 'MEXICO) ) 
« « 

: T ; .(FOR: THE A'0008 / J FIND : PERSON. (FIRSTNAME 'BILL)) 
T ; (FOR:* ALL AOOIOV . - ^ 

~~ ^ (FIND: TRIP (TRAVELER AQO08) 

. ■ .(DESTINATION A0009) 

» « (TIME (BEFORE NOW-)')) 

' ' : T ; (OUTPUT: (GET:^A0010 1 TIME] 

The interpretation is, a quantified expression which can (almost) 
±>V directly executed. Translated it says, roughly: ; - > 

Find the location .v/hich has the country % name ' 
"Mexico" arid call it A0009. Find the person whose first* 
• naffle, is^ "Bill" and* call Kim A0008. Find in turij, all. - • 
tr ips* whose traveler-is A0008, wtjose' , destination' is 
A00t)9, v and which occurred prior to now. Afe each is ' . 
enumerated,' label ijt A0010 'and output its specific 
time. 

• " f 

* > 
/ . * 

The T's which app'ear in the* interpretation 'are there in place of 

%r * • 
potential restrictions which might have been applied to the' 

location, person, or trip classes being snatched. 1 V 

< Before the ■ interpretation can be executed it ufiderqoes ah 
op,timi.zation (see Reiter, 1976) '-and modification *to match the 
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'data base representations. in this example .there is no entity 
called "LOCATION in^ the semantic net#or r k dat;a base which serves 
the purpose implied in the '^interpretation. There are' cities, 
states, coasts, countries", etc. and- there is a link called 
"LOCATION", but f no concept with instances a|s we might expect. The 
interpretation - thus references a fictitious node. The optimizer 
recognizes this and modifies the interpretation so that a search 
is done among all the possible "locations'". . In HWIM, this 
modification is specific t<> the' f i<^tTtious nodeT^ha^^as beerT 
referenced. More recent knowledge representation systems, such 
as KL-ONE would f ormalize, f the ability to fornw abstractions such 
as . "location" from, the more specialized concepts (Brachman, 
Bobrow, Cohen, Klovstad, Webber, & Woods, -Note 3).- 

The optimized interpretation is placed on a queue «of 
tOTbe-epcecuted commands. The queue represents a 'first^step 
towards a "demand model of discourse" (see next section). In 
principle <: it can contain previous queries or commands 4 from the 
speaker a£ well* as- system initiated 'con^ands. ' However/ most 
utterances? translate into single commands which ars directly* 
executed. , „ 

, • . • . . ■ .; , ■ v .... 

-The (FOR:"—) expression above* is t a LISP form which can. be 

f 

evaluated directly. The FOR: function manages quantification in 

the expected way'. Here it looks for a unique location and a 

._• • . , / 

f , * 

13 • 
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unique person w&ieh match the respective! descriptions. There is a 

: ' • ' i 

single place whose country name is "Mexico," i.e., the country, * 
Mexico. However,' there are 8 items in the' data base representing /t 
people whose first name is "Bill." Given the apparent 
inconsistency between the data base and the command, FOR: is 
forced to take the initiative of the dialogue and ask the manager 
for help. The response generation program produces the question: 



Jo=you — mean — Billy" Diskin f Bill HuggAs,* Bill 



Levison, Bill Patrick, Bill Plummer, Bill Russell, Bill 
Merriam, or Bill Woods? * $ % 

When the system asks a question (a^in tfiis example) it 
expects an answer. This means that otherwise "incomplete" 
utterances may be accepted. Here the parser will allow a name as 
a complete utterance. In this context, the person types a second 
sentence: 

Bill Woods. . v 

This sentence undergoes a pre-pass to produce the word list 
■ (BILL WOODS) * 



which the parser interprets as 



(! THE A0011 PERSON ( (FIRSTNAlW BILL) 

(LASTNAME WOODS))) 



it e 



s> 
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The, optimized interpretation is ' 



. . (Iota i aoqii,,/ (find: persqn 

f\ * /' " (FIRSTNAME 'BILL) 

' ' * , (LASTNAME 'WOODS)^: T ) . ' 

m » ' " *■ 

• * * 4 

This interpretation us.es the * IOTA operator,- a function that 

9 n « • * 

returns the one itkm in the'* class defined by the (FIND: - -) 
expression which meets the restriction, T (i.e., no restriction). - 
•In 'this case there is .exactly one item matching the description, 
namely BILL WOODS . ' 

« * * -»V * 

' At this point, we are -in the middle of what Schegioff (1972), 

has called an "in-sertfon sequence." In the process of answering 

the user's question, 'the , system began executing a* 

FOR:, fexpressron. -'Evaluating 'the FOR*k expression required 

information that was obtairiable only by' asking the f user a 

question. HWIM's .question tg the user, plus* the user's answer 

constitute an igseition sequence with respect to the primary 

question-answer sequence. The execution of the *FQR: expression 

is " suspended for the duration of the 'insertion sequence, but cah 

% -*- . \ 
be resumq<| when* the needed information is processed. Having now 

a unique ".' DESTINATION and a uniaue TRAVELER, FOR: can find all 

trips whose^T-IME" is (BEFORE NOW) and for each,- output its TIME. 

Finding '^the trips, checking TRAVELER, DESTINATION, and TIME 

« * 

can require cons ide'r able- -search' in the data base. For example, 
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the DESTINATION of a TRIP is computed from the. DESTINATIONS pf 
, its component LEG/OF/TRJPs. The DESTINATION of 3 LEg)^/TRIP may 
have to' be ^computed from the PURPOSE of the LEG/OF/TRI^Xe:g. , 
attending^a specific ^conference. . Information . about trip< 
budgets', conferences, etc. is never ass'unred to be complete since 
it is not £o ^ in the % real .world., Furthermore, the ran^e^ 
askable questions, precludes* cdmptating in advance all the implicit 
information. * (The procedures .that^ do these computations are 
giscuss£d.in Woods et al., Note 9.) In this example there is 
'only one trip that ^ntjatches the description and 'its. time is 
printed out by the response generation programs: 

Bill Woods's trjp ^rom Bostonr Massachusetts to 
-Juarez, Mexico was from October 25th to 17th, 1975. * 



Discourse Model', 



In any conversation there are' „many • forces operating to 
determine what will be said next, including the ^memories, the 
intentions, and Jthe perceptions of the participants in the 
conversation (Bruce, 1980, Note 4). A discourse model^ is an 
idealized # representation of such forces as t£ey are used by a, 

speaker in. constructing an utterance and by .a listener in 

* * **■ 

understanding one. . (See also Reic^nan, 1978; Stansfield, 197 4; 

Deutsch, 1974; Sidner, Note 8). , 



16 



• / 



A 



.1 



| • ..\ . ' * HWIM: A Computer Model 

« . \15 

The discourse ^context ,is an important factor in even the 
restricted domain ^of personi-computer communication, but in many 

. interactive natural language understanding systems there, is -no 
explicit "discourse model*. '-Most" artificial problem domains are_ 
deliberately cqnstructed so tHat only a few interaction modes are 

• sensible. ^ For example, th,e person may only jask questions; the 

o , 

'syste'm may 'only answer the que'stions. This greatly simplifies 
the determination *of the speaker's intent, arid hence, the full 
understanding o£ the utterance. There* is also ' no need to 

maintain a data base of 'participants in*a conversation (together 

> 

with their presumed purpose, • attitudes,- models*' iof the Vorld', " 
*etc.) when the only other part icipant .in the ~ conversation is "the 
USER.". Even in those cases where the problem definition permits 
*a general discourse dnd ttoe systefh is ''able to' cd]pe with that 
generality, the' means o\ coping are rarely delineated (Bruce, 
1975b>. / - J - 

Incremental simulations (see Woods & Makhoull, 1973) of 

dialogues between a travel budge't 4 manager and HWIM (a system 

• \ * 

builder played the part of an ideal HWIM) suggested "..al discourse 
model in whi-ch, at any point, the manager could b£ 3e4n as being 
in one of several states, e.g. , trying to determine * the 

* . * . • : t 

consequences of a proppsed trip, examining the 'state of the 

budget, br entering new trip* information. We considered! several- 
ly " 



formulations of a discourse model in which these states Would be> 



er|c 
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•made explicit /and used to advantage. 



One such formulation involved the concepts of modes of 
interaction and^ intents v (Bruce, 1975c). An .intent is the assdjnea 
purpose behind an ^utterance, "patterns of intents such as, , • ' 



ERIC 



user„-enter-new- information. 

sy s tem-po i n t-ou t -con t rad ic t io n 
* . ' user-ask -quest ion 

system-answer-question 
user-make-edit ing-<feange 
syatem-con firm-change , 

• - - . X * 

constitute the modes of interaction. 

♦ / 

.An augmented transition network (ATN) ' grammar was used to 

^represent some c^f the common modes of interaction found in travel 9 

* budget management 'dialogues . Then Na modified ATN" parser 'was 

written that steps; through the grammar on the basis of the input 

v r • ' \ t 

sentence structure and the then-cur r en, t state of thfe data base. 

, At any given^ state the parser can predict the most likely next * 

intent and hence such things, as the* class of likely verbs for the 

next utterance. This model was not used in the final version of 

the system, primarily because it was too rigid to function alone 

J ^ 
as*a discourse model. 

Another formulation of the <user/a is course moddl irivolves the 
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notion of demands made by participants in the dialogue. (3oth of 

t 

these formulations are discussed more fully j^n Bruce, 1975b) . 
Demands include J such things as , unanswered questions and 
contradictions .which have been pointed out. This latter 
formulation allows us to model how one computation of a response 
can be pushed down, while a whole dialogue takes place to obtain 
missing information, and how a computation ca-n spawn subsequent^ 
expectations or digressions. Some elements of this de^nd* model, 
together with other aspects of the discourse model, are 'explained 



below: 



(1) Demands : These are demands for service of some kort 
made upon the system-by the u§er or by the system .itself. An 
active unanswered question is a typical demand with high 
priority. The fact that some questions cannot be ' answered 

without more inUbrmation ' leads to an embedding of modes of 

*■ » * 

^interaction. •Demands of lofrer priority include such things as a 
notice by the system that the manager is over his budget^ Such a 
notice might not be communicated until' after direct questions had 
been answered. > * • 



* ' v •* 

(2) Cou n te r-d ema nd s : ' These are questions the system has 
ft * 
explicitly or implicdtly asked tfie user . (e.g. , "This trip ,will 

put you over , budget" implicitly ' asks the user to review the 

•budget, canceling budget .items or adjusting- cost estimated). 

-* . lb • 
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While it should- not hold on to these as long as it does to 

demands, nor expect too strongly that they will- ^e met, the 

. . <** 

system can ^ reasonably expec£ that most counter-demands will be 

resolved *in some .way. .This is' an ' additional influence" on the 

discourse structure. r ~ * . > • 

t 

(3) Current discourse state ; The discourse area of the data 
base also contains an assortment of items that define the current 
discourse context, .including: 

X 

o LOCATION, a pointer to the. current location of the 

9 v m 

speaker, e.g., the city . V. . 

o TIME, a pointer to the current time and date 

V 

k 

o SPEAKER, a pointer to the current speaker 

o the last mentioned person, place, time, tVip, 
budget, Conference, etc. 



i 

\ ! 



There is also a representation of the Current TOPIC (see Figure 
3) ^ This is the active focus of attention in th* dialogue. ■ It 
could be the actual budget', a hypothetical , budget , a particular 
trip, or/a conference. , The current topic is used as an anchor 

» 4 

point for resolving definite references and deciding how much 
detail to give in responses, it can also generate certain modes 
of -interaction. For example, if the manager says "Enter a tr.ip," 




J. A discourse state. 



» 
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the 'system notes that, the current - topic' has changed to an 
incompletely described trip. This results in demands that cause 
standard , fill-in questions to be asked/ -If \he manager wants 'to 



complete the trip description later/ then ttfe completion of the 
'trip description becomes a low priority demand. 

The system has a primitive one-queue implementation o£ ' the*" 
"demand model." This queue contains executable procedures which 
represent the speaker's previous queries and' cdmrnands as. well as 
commands initiated by the system to examine the pcmsequences of 
-its actions, give information to the user/ or* chepk for, data basa ~ 
consistency. These procedures are related by functional* 

dependencies ^and relative priorities.. The majof .t^ypes pf demands 
are* the following: 0 ^ A \ J /J < ; 




o % DO: me^ns execute the specified command. 



O' TEST: -means evaluate the form and answer "no", if Nik' 
or "-yes" otherwise. „ j 

p RESPONP: means give the user some information (which 
may or iriay not be part of an 'ajiswer to a direct 
que'ry) . 

<r PREVENT; ' means monitor; for a subsequent possible 
action and block its .normal, execution (as in "Do not "".<* 
allow jnore than three trips to Europe. n \. - .« 
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<3 • ' ^ ' • Understand ing 



• The process of understanding written natural language 
requires the integration of diverse" knowledge sources. An optimal 
process must .apply various types of* knowledge .in a balance that 
avoids over r or under-constraining the set -of potential account^ 
of the input. * ' • 

Ultimately the understanding process should produce a 

meaning representation JEor the natural language input, whether 

the- input be written or spoken*. This representation ftiay be 

i 

produced directly or via some number of contingent knowledge 
structure's (Bobrow & .-Brown, 1975"). The exact form of 'these 
-structures .varies firom one system to*" another, but typically 
includes, such things as'parse trees. 

In HWIM, integration of knowledges sources for liriguistic 
^jpr^cessing - is . accomplished by means of -(1) an ATN grammar (see 
Figure *4) that incorporates much'of the syntactic, semantic, . and 
pragmatic '^knowledge . the system, *(2) a semantic network (see 

^^ash-webber, 1975)r 'that' represents remaining linguistic and world 
. knowledge, and (3) tests on arcs in the grammar us^d to, iink the 



two^ representations. These structures are used to produce 'the 

: ' ' * • 

account of. the input that serves . as a representation of its 

* ' < \\ 

meaning. 



1. 
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«Fig.ure 4. A 



small 'portion of 
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x The decision to include semantic knowledge - in a. grammar has 

'both^its merits and^ its defects. On the one hand there** is an 

obvious- advantage in t>erms of efficiency when semantic knowledge 

*can be appliedWearly on in the most direct way to constrain * the 

possible interpretations^ of an utterance. Moreover, one avoids 

the seemingly unnecessary step of building a syntactic, model, 

such" as a parse tree, for an utterance'. AM of this points to 

the desirability of a # unified linguistic processor as exists , in 

HWIM (see also Erman, Hayes-Roth, Lesser, & Reddy, 1980;. Burton & 

* * <fe . * * 

Brown, Note 5). On *the other hand, the fact that we could 

incorporate semantic and pragmatic "knowledge in the grammar*' and 

use that knowledge as if'it were fundamentally no different f^om 

syntactic knowledge was a consequence of having a limited - domain 

of" discourse. No computer natural \language system Has' yet 

. . ^ i — 
approached the variety and cbmplexity that natural human 

' communication exhibits. As' semantic and pr-agmatic knowledge 

becomes more 'complex, more/ fluid; and more x intricately 

interconnected, it not clear to what exte'nt* the pragmatic 

grammar approach will' work. 

e m 

Despite the inclusion of non-syntactic knowledge, the 

** 

pragmatic, grammar is no't a' complete knowledge source* by itself 
for linguistic processing. It is, however, closely linked to the 
semantic network via tests on the arcs^ These tests allow the 
grammar to capture more volatile constraints on the* input, such 
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^as. those provided by the discourse model or the factual data* 

0 + 

base. .For example : ^ * 

, " • v . 

. . o Is. "Jerry Woods" a valid .name-?-?- 

- o .is "San Francisco, .Cali fornia< alalia place? 

^ c o What words can go with " "speech 1 ' as a project 

descriptor (e.g., "understating")? ^ * V 

' / " * 

o Does "What* Is "the registration fee?" make .sense in , 

•♦ this context? 

o. Does "How much is in the budget?" make sense in this 1 
context? , a»- 

£ ■ - ■ 

o Doe^'When is that meeting?"' -make sense in this' 
context? ■ ' r\ 

, o Does "November lJ5th" make sense in this context? 

• • ; ■ * • • : ; 

With the grammar encoding semantic and pragmatic, as well as 

syntactic information, it is. possible for, £he parse* to build a 

. • *\ 

procedural' "meaning" representation as well, as a purely". syntactic 

one. The meaning representations are in a command language whose 

expressions are atomic, consisting of. a^ "functional operator' 

applied to arguments, o'r compound, making' use of a quantification 

operator (FOR: ) applied to another expression. Th^ operators in. 
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atomic expressions specify operations to be performed on^the 'data 
blase or interactions with the user. 

The" parser builds interpretations by> accumulating in 

registers the semantic head, quantifier, and links of the nodes 

■ -* 

being- described in the sentence (see Woods et al. , Note 9, for a 
more- complete description). For exampJe, the sentence 

I^wiic go to the ASA meeting. 

yields an interpretation in the command language (Section 
COMMAND) of the form * ' 

(FOR: THE A0018 / (FIND: MEETING (SPONSOR ' ASA).") : T ; 

'(FOR: 1 A0019 / (BUILD: TRIP 
v ' " _ 

(TO/ATTEND A0018.) 
W (TRAVELER SPEAKER) 

(TIME (AFTER NOW))) 

: T ; T) ) 

This interpretation is i built up in the following way. A 

, constituent describing, a per s on^is^foiind at— the — Wa-r-t— -of- — ttre - 

* sentence. The* parser transforms, the pronoun, "I" into, the 
link-node pair « (TRAVELER SPEAKER) and returns' this as the 
interpretation of that constituent. The word "will" adds (TIME 
(AFTER NOW)) to the list of link-node pairs being accumulated. 
(The grammar does not accept constructions "like "will have gone," 
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so "will" can always be interpreted'as marking a future event.) 
The word "go," in th,e context of our travel domain, vindicates 
that a trip is being discussed. Next, the constituent' "the ASA 
meeting" produces 

(TO/ATTEND (I THE Y MEETING ( (SPONSOR ASA)))). 

(The ! indicates that a 4 FOR: expression 'will have to be built as 
part of the interpretation.) The top level of the grammar has 
thus accumulated the link-node pairs 

(TIME (AFTER NOW) ) ^ 
(TRAVELER SPEAKER) 

(TO/ATTEND (! THE Y MEETING ( (SPONSOR 'ASA))))^ 

with the semantic head TRIP. The appropriate action (in this case 

a BUILD:) is created, to produce . .* 

& ~* 

(BUILD: TRIP . 

* (TO/ATTEND (! THE Y MEETING ((SPONSOR, 'ASA))')) 

(TRAVELER SPEAKER) 

(TIME (AFTER NOW))) * 

* * 

« 1 

Finally, the necessary quantif icational expressions are 

expanded around the BUILD: express ion* 



- ' Response Generation . * 

Once a satisfactory theory £or an utterance has been 
constructed, it must be acted upon by the system. Regardless of 
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the # type of action taken, some appropriate response -should also 
be made. From the speaker's point of view. the response should' be 
explicit, concise, and easy to. understand. W response may 
affect v the speaker's way of describing entities" in th-e^dc^ain as 
well as illuminate the capabilities and operation of the system. 

• The effects of generated responsesffcre many. , In a domain 
such as travel budget management there are many objects without 
% stand-ard names, e.g., "the budget item for two trips to a West 
Coast conference". Names chosen (constructed) by HWIM provide a 
handle for the manager to use -and thus < strongly influence the 
ways the objects are referred to- subsequently. Responses can 
also indicate the capabilities jof the program. ta ireful 
construction of responses to exhibit' exactly the' knowledge 
structures handled" by HWIM gives the manager the legitimate 
confidence to pursue just the. . paths , which rely on those 
structures. A Response can also show the system's focus of 
attention. The manager 'can thus get -a better idea, not only of 
the facts/ 'she or he seeks, but also of how well the system 
understanding and where .rnore^ clari ficatiort may. be useful. 

0 ■ ( Types of Knowledge used _in HWIM ' 

& order to carry out; its role effectively, HWIM -must have en- 
large amount ol knowledge in a readily accessible form. This* 
knowledge is needed- for understanding bot;h spoken* utterances- <md- 
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written sentences, carrying out commands, answering questions, 
and generating responses. HWIM's success in performing these 
tasks is evidence that the types of knowledge embodied in the 
system 1 are sufficient foKTiatural communication, at least at the 
level shown by sample dialogues. The' history of the development 
of HWIM (Na^h-Webber & Bruce, T976) suggests that each m category 
of knowledge is also necessary for natural communication to take 

.0 

place. The kinds of knowledge used by^HWIM can be categorized as 
follows: 

o Acoust ic - Phone t ic forms — Knowledge of phonemes and 
their relation to aqoustic parameters. 

o Phonological rules — Knowledge" of phoneme clusters t 

and pronunciation variations describable by rules. * 

* < 

o Lexical forms — Knowledge "of words, ' their 
_ inflections, parts of speech and phonemic spellings. 
For* example, "entered" is the past form of "to 
enter". - - ' 

* o Sentential > forms — Knowledge, of grammatical* 

structures at the phrase, cflau.se, and sentence 

V level. For example, a sentence may start with a 
dommand verb, such as "schedule.. 11 ' 

' o ^Discourse form —Knowledge of „ idealized- discpurse, 

30 • 
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.g.,- what type of utterance is likely to be 
produced in a given sbate of the discourse.^ 

X- • ' A • ' * 

Semantic — Knowledge of how words are related and 
used and structurally conveyed meaning. This is 
uSed in parsing and in constructing responses*. For 
example, the benefactive case for "schedule -should be 
•filled by a person who will be taking the trip being 
scheduled. • - 

World — Knowledge of specific projects/ trips, 
budgets -and conferences. Whereas HWIM's semantic 
knowledge is essentially fixed, its world knowledge 
changes every time a trip is taken or money is 
shifted frpm one budget item to another. „ - 

On - Going discourse — Knowledge of the current 
discourse §tate, e.g., v the current f topic and' the 
objec ts available for anaphoric reference. This 
knowledge is used to rela^ constraints ^in the 



f 



pragmatic grammar. For example, if a conference is. 
under discussion, then "What is the registration 
fee?" is a meaningful utterance. If nbt, then the * 
speaker would have to say something like, "What is 
the registration fee for the ACL" conference?" 
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o Parameters of the communiaative* situation — HWIM's 
m 

operation is a function of the modality in which it 
operates; - speech ot text input .and speech or text 
s output. In general, communicative situations can 
vary along .a numberN^f^dimensions (see\ Rubin, 1*980) 
and a si^ccessful communicator must- use knowledge of 
the situation to' interpret and respond 
appropriately. 9 

o User—Knowledge about each po^iblB- travel budget 
manager, what groups they belong, to, and what they* 
may know about the 'data base. For, example, only 
certain users may be allowed to modify budget 
totals.,- 

o Self -^Knowledge about the system's own knowl-edge 

/ (often called "meta-knowledge")'. .One example is 

that data dn trips and budgets is marked to indicate 

-whether it came froifr the travel budget manager or 

was # computer by HWIM. Anothey is that HWIM knows 

** / 
what elements constitute, a complete description of 

* j\ * 

any object, such as a trito/' Thus it knows when its 

ow$ knowledge is incomplete> < * 



V • • r • • 

o Task — Knowledge of the 'task domain, e.g., that a 
manager is concerned with maintaining an accurate 
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record of all trips, 'taken' or planned, and with 
staying within the budget. 

o Strategic — Knowledge* concerning ''the representation 
and ■ us£ of other knowledge. This knowledge that 
would be needed in even a "blank" 'system/ 1. e. , one 
that had no word- or domain-specif ic knowledge. 

Conclus ion 

^One reason for building a computer model of language 
understanding is that in the course of designing and debugging & 
computer program, one must resolve theoretical questiongFabout 
details of the process that can be glossed over in less 
procedureror iented models* For example, designers of computer N 
models of speech act generation (Cohen & Perrault, 1979) have 
increased the precision of speech act definitions. A second 
reason for computer models is that the task <5f the system can be 
defined to require complete use of language (within a restricted 
domain, of course) . . Thus,* one can see. what components, each 
representing .aspects of language theqry, are needed-and how they 
.might interact. Winograd's (1972) blocks world program, a system 
Which accepted typ'ed questions,' commands, and assertion^^ 



performed actions, ~and generated natural* language responses, 
could-* be put in the latter category. There* is value in .both of 
these approaches; in fact, they tend to 'complement each other," 
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^ >s ^ ' * 
with the" second showing what can and cannot be done and the first 

addressing specific design questions. 

1 0 

jHWIM is in both categories, but its principal value probably^ 

lies more in ' the second thcup the first. To some extent,- it 
represents. a synthesis of a number of lines, of research in areas 
such as parsing, inference,- data base design, and generation. It 
shows what can be done in a f airly ./^pomplet^ system' that 
understands 'natural language (typed or spoken),' answers questions 
and performs various actions, and responds^'in natural language 
(typed and spoken) . 

Prbblems that arose in the design of HWIM 'were precursors of ■ 

those tihat are central issues in AI research today. jPor example, 

the speech act' issues for HWIM are similar to those studied "by 

Cohen (Note 6), Cohen .and Perrault (1979), and; Allen (Note 1)» ' 

/ 

Questions 6f knowledge representation closely related feo those 
f^ced in HWIM have been .pursued by Bobrow n and Winograd* (1977), 
t Brachman (1979), and Fahlman (1979). (Also ?ee Brachman &• Smith, 

* 

*1980) . Language generation in a discourse context similar tQ ' 
HWIM s *has been studied by McE)onald (Note 7) , for instance.* 
Finally/ the issue of interactions among syntax, semantics, ^nd 
pragmatics is crucial in the work of many,' ingluding Schank and 

» 

Abelson (19 77) ,• woods (1980a), and Bobrow* (Note 2) . The 
characteristics of HWIM reflect the goal of natural communication 

34 • '• r ' 
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1 ' 

. << ' 

between a person and a computer assistant. Even in its limited'' 

* — 

domain,, it illustrates the extent to which natural communication'' 

depends upon diverse kinds of knowledge in both communicants. 

• 1 

The structure of HWIM can, provide . a useful framework for 
obtaining a better uncle f standing of^tfatWal communication. , 
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APPENDIX A 



The Structure of HWIM 



The structure of HWIM is shown in Figure 1. Components .of 

the system are shown in ovals with thick arrows representing flow 

* ♦ 
of control between components. Major data structures^ are shown 

in clouds with thin arrows indicating data flow. The system is 

implemented on TENE&, a virtual memory time-sharing system for 

the DEC PDP-10. Each component exists in a separate TENEX 

process (called a "fork"). Among the reasons for this 

multiple-pfocess job structure .(see Woods, et al., 1976) are that 

most of the components ar^s so large, in terms of program and data 

structure requirements, that they cannot exist in the same TENEX 

address space with another component. . * w 

* 

The Trip component is the one primarily discussed in 'this 
paper. It controls the system and is the major component for 
acting oh and 'responding to an utterance.* If it 4 " is given -a typed' 
input, it calls Syntax to parse the sentence. Otherwise it calls 
.Sp^pch Understanding Control. •■ * ■ * • ^ • 

/ Most of the other components perform dingle functions: 
Dictionary Expander expands a dictionary of Ipaseform 
pronunciations using a set of phonological rules (applied once at 
system loadup time) . Real Time Signal Acquisition - (RTIME) ■ 

[ V. 
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digitizes and stores the speech signal. Signal Processing (PSA) 
converts the speech signal into a parametric^ representation. 
Acoustic-Phonetic Recognizer (APR) operates on the parametric 
representation of the speech- signal to produce a set of phonetic 
hypotheses represented in the segment lattice. Lexical Retrieval 
searches the expanded , dictionary and matches pronunciations 
against portions of the segment lattice. Verifier generates an 
idealized spectral representation of a word (or wordsK, using a J 
speech synthesis-by-rule program and matches^it against a region 
of the parametric representation of the speech* signal. Syntax 
judges the grammat icali ty * of a given word sequence; predicts 
possible extensions at each end of the' sequence; builds a formal 
representation of an utterance. Interpreter (present in an eairly 
version only) takes a syntactic representation of an utterance 
3nd builds a' procedural representation of .the meaning. Talker 
generates speech from phoneme-prosodic cue sttings. 



